An Evaluation of the DEC Memory Channel Case Studies in Reflective Memory and Cooperative Scheduling

With the rise of clusters as a vehicle for very-highperformance computing, an increasing emphasis is being placed upon the communication interface between each processor and the underlying network. And while many studies have explored the design of both traditional send/receive network interfaces and shared-memory designs, a third alternative — reflective memory — offers a significantly different design space and provides its own, separate benefits and challenges. In this paper, we investigate Memory Channel, a high-speed reflective-memory network, by implementing MPI above Memory Channel. Our results show that Memory Channel is a powerful substrate for message-passing programs, but that significant obstacles remain to building truly largescale message-passing software above this network. Finally, we investigate time-sharing of a Memory Channel-based cluster and demonstrate the need for event support at the hardware layer.